Repository-level data-table linting for data-manager / reference-data bundles - #23229

Two provable hard-error checks over RepositoryDataTables:
- MissingLocFixture: a configured table references a loc file that
  resolves to nothing (sample fallback still counts as found).
- LocRowShape: a non-comment loc row can't supply every declared
  column index (too-few-fields / wrong-separator), reusing the
  row-shape errors TabularToolDataTable captures at load.

Driven via lint_ctx.lint so each is skippable by name. Fixture-based
tests (no mocks) cover clean / missing / broken / sample-fallback /
skip cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Cross-component name validation over RepositoryDataTables:
- ManagerTableConfigured: a data manager populates a table that no
  local tool_data_table config (nor known-external table) defines ->
  error (broken producer contract, Planemo galaxyproject#706).
- ConsumerTableDefined: a literal from_data_table reference resolves
  to no local/known-external table -> warn (may be supplied by core
  or another installed repository).

Only literal, fully macro-expanded names are checked; names still
carrying $/@/{} markers are treated as not-checked, guarding the
tools-iuc#5003 false positive where @IDX_DATA_TABLE@ looks
unconfigured but resolves. external_table_names suppresses both.

Fixture tests (no mocks) cover clean bundle, unconfigured manager
table, externally-supplied table, unknown/known consumer, and the
non-literal guard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

OutputRefValid: a data_manager_conf output_ref that names no output of
the expanded manager wrapper is a hard error, listing the real declared
outputs for the fix. Only runs when the wrapper actually resolved --
ManagerDecl now carries wrapper_resolved so an unparsed wrapper leaves
outputs unknown (not-checked) rather than demonstrably missing.

Fixture test covers a real-output ref (clean), a bad ref (error), and
the unresolved-wrapper guard.

Duplicate-column-name and conflicting-schema detection deferred: a
conflicting same-named table config raises inside
TabularToolDataTable.merge_tool_data_table (the 26.1
assert_data_table_consistency path, outside planemo's <26.1 pin), and
duplicate column names collapse in the parsed columns dict, so both
need model work beyond this slice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Unlocks the two checks deferred from step 4 via a pre-merge raw view of
each <table> element (RawTableDecl: declared column names kept with
duplicates, separator, comment_char, source), since the parsed columns
dict collapses duplicate names and the loader *raises* when merging
same-named tables with different columns.

- DuplicateColumnNames (error): a <table> declares a column name twice.
- ConflictingTableSchema (error): a table name is declared with
  differing columns/separator/comment across the bundle.

build_repository_data_tables now skips loader enrichment when a column
conflict is present (the loader would crash) and reports it from the raw
declarations instead; table names come from the raw view too, so
cross-component checks stay robust. configured_table_names unions the
loader-enriched and raw declarations.

Fixture tests (no mocks) cover duplicate columns, conflicting columns
(assembly must not crash), conflicting separator, and a clean repo; a
direct-loader check confirms the skip guard is load-bearing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Review-driven cleanups on the repository data-table bundle:

- _raw_column_spec: delegate the name->index map to the canonical
  TabularToolDataTable.parse_column_spec_element (added by 94a027c);
  keep only the ordered-names-with-duplicates list DuplicateColumnNames needs.
- Drop unused TableDecl.instance + RepositoryDataTables.table(); fix the
  stale "reused by downstream" comment (schema fields kept as honest forward
  surface for bundle-completeness).
- Use xml_tree (house standard in the tool linters) via _tool_source_root
  instead of the equivalent root attribute.
- Tie _has_column_conflict's docstring to the canonical loader/merge check.

Add lint_repository_data_tables_bundle(lint_ctx, repo_root, ...paths...): a
discovery-driven entry point that builds the model and lints it, dispatching
the assembly phase through lint_ctx.lint so skip/assembly-failure diagnostics
print. Lets repository linters (Planemo shed_lint) do discovery + one call.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

- Restore RepositoryDataTables.table(); it is covered by a step-1 model test
  (test_repository_data_tables.py), which the earlier removal missed.
- test_list_linters: 147 -> 154 for the 7 repository data-table linters, and
  import bundles.lint there so the census is deterministic (matches how the
  test already imports every linter module).
- Silence 7 Liskov [override] errors: the bundle linters intentionally target
  a RepositoryDataTables model, not a ToolSource.
- cast the tool_data_table_confs list at the ToolDataTableManager call to work
  around list invariance (str paths vs list[str | PathLike]).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Relocate data-table bundle discovery (find data_manager_conf /
tool_data_table_conf, walk consumer tool sources) out of Planemo into
find_and_lint_repository_data_tables. Add galaxy-tool-data-lint
console_script over it so a data-manager/reference-data repo can be
linted standalone (--skip/--json/--fail-level). CLI tests over fixtures.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Real shed repos ship tool-data/<name>.loc.sample and materialize the real
.loc on install. The loader's own .sample fallback strips the tool-data/
subdir and only looks in tool_data_path root, so it misses these
(found=False) -- MissingLocFixture would then falsely flag nearly every
reference-data repo. Add LocAsset.sample_backed (sibling <ref>.sample or
tool-data/<basename>.sample) and only flag references that are neither
found nor sample-backed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

list_linters() now imports the repository data-table linter module itself
so the registry is populated deterministically wherever it's called (incl.
planemo's --skip name validation), instead of relying on a side-effecting
import in the tool-linter test. Drop that import; count stays 154 (147 tool
+ 7 repository linters). Add a README describing the lint fixture repos.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…efault

find_and_lint seeds external_table_names with the tables Galaxy core / a stock
data manager supply on essentially every deployment (all_fasta, fasta_indexes,
__dbkeys__), so ConsumerTableDefined does not warn on ordinary from_data_table
references to them. Callers can extend the set for repository-specific suppliers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Warn on a .loc / .loc.sample that is empty and carries no comment
describing the column format. A dataless file with a documenting comment
is the accepted convention (a data manager fills the rows on install), so
only empty-and-undocumented files are flagged; warning, not error.

The check reads a fresh repo-tree walk (model.loc_files), not the resolved
loc_assets: the empty .loc.sample files this targets never resolve into
loc_assets because the loader's .sample fallback misses the shed tool-data
layout. Across tools-iuc this flags 32 files (18 test-data/*.loc, 14
*.loc.sample).

Give the clean fixture's 0-byte test-data/all_fasta.loc a format header so
the canonical happy-path repo passes every linter.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add core_table_consumer fixture (from IUC data_manager_bwa_mem2_index_builder):
defines bwa_mem2_indexes locally, consumes core all_fasta via from_data_table.
find_and_lint seeds all_fasta/fasta_indexes/__dbkeys__ so the core ref doesn't
warn; without the seeding it does (non-vacuous guard).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>